Robust Calibration of Camera-Projector System for Multi-Planar Displays

We present a robust calibration method for aligning a camera-projector system to multiple planar surfaces. Unlike prior work, we do not recover the 3D scene geometry, nor do we assume knowledge of projector or camera position. We recover the mapping between the projector and each surface in three stages. In the first stage, we recover planar homographies between the projector and the camera through each surface using an uncalibrated variant of structured light. In the second stage, we express the homographies from the camera to each display surface as the composition of a metric rectification and a similarity transform. Our metric rectification algorithm uses several images of a rectangular object. In the third stage, we obtain the homographies between the projector and each surface by combining the results of the previous two stages. Inconsistencies appear along the boundaries between adjacent surfaces; we eliminate them through a process of iterative refinement. Standard techniques for recovering homographies from line correspondences and performing metric rectification are very sensitive to image processing outliers. We present robust algorithms for both tasks, and confirm that accuracy is maintained in the presence of outliers, both in simulation and on our interactive application that spans a table and adjacent wall. Our calibration method enables users to quickly set up multi-planar displays as they are needed, using any available projector and camera. These displays could be applied to visualization tasks in medical imaging, architecture and geographic information systems.